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PREFACE 

This  Memorandum  presents  the  results  of  computational 
experience  with  certain  variations  on  the  "simplex  method" 
of  solving  linear  progriimmlng  problems.  The  purpose  of 
this  study  Is  to  establish  a  basis  for  comparison  of  the 
efficiencies  of  various  procedures  in  computation.  Re¬ 
search  on  linear  programming  is  conducted  as  part  of  The 
RAND  Corporation's  basic  studies  In  mathematics. 

This  Memorandum  should  be  of  particular  Interest  to 
those  concerned  with  linear  programming  and  of  general 
Interest  to  other  mathematicians  and  computer  specialists. 
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SUIWARY 

This  Memorandum  summarizes  the  main  results  to  date 
of  the  SCEMP  Project  (Standardized  Computational  Experi¬ 
ments  In  Mathematical  Programming),  Involving  the  solution 
of  nine  linear  programming  problems  using  30  variations 
of  the  simplex  method.  The  statistics  collected  allow 
comparison  of  most  of  the  variations  which  have  been 
proposed  In  recent  years  and  l.ndlcate  the  Important  fea¬ 
tures  In  the  efficiency  of  linear  programming  routines. 
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I.  INTRODUCTION 

There  are  many  ways  to  solve  linear  programming 
problems.  The  earliest  of  these,  Dantzlg's  "simplex 
method," is  the  most  widely  used  and  no  equally  ef¬ 
fective  alternative  Is  available.  Many  variations  of 
the  original  simplex  method  have  been  proposed  In  the 
last  few  years.  Computational  experience  seems  to  us 
the  only  way  to  properly  compare  the  computational  ef¬ 
ficiencies  of  the  variations;  their  behavior  depends  so 
strongly  on  features  of  the  process  which  cannot  be 
known  In  advance  that  a  priori  estimates  of  their  ef¬ 
fectiveness  Inspire  little  confidence.  The  purpose  of 
the  work  reported  here  has  been  to  compare  some  of  the 
outstanding  variations  with  each  other  In  their  work  on 
actual  linear  programming  problems,  and  to  set  some  bench 
marks  against  which  other  procedures  may  be  measured. 

Under  the  ‘■Itle  of  "SCEMP"— Standardized  Compu¬ 
tational  Experiments  In  Mathematical  Programming — this 
work  originated  In  i960  at  a  meeting  of  the  Linear  Pro¬ 
gramming  Committee  of  the  SHARE  organization  (IBM  704- 
709-7090  users  group),  when  It  was  suggested  that  some 
of  the  linear  programming  routines  then  forthcoming  might 
serve  the  task  of  evaluating  the  procedures  that  had  been 
discussed.  The  Committee  maintains  a  file  of  problems 
from  which  those  used  here  were  selected;  they  are  de¬ 
scribed  In  detail  In  the  next  section.  A  set  of 


ntatlstlc-collecting  routines,  modeled  on  an  all-ln- 
core,  FORT  RAN- coded  linear  prograjnmlng  routine  for  the 
IBM  70^  and  7090^^^  was  coded  and  served  as  the  basis 
for  the  computer  routines  used  In  the  present  tests. 

(The  routines  and  the  output  of  the  tests  have  been  re¬ 
tained  and  can  be  made  available,  but  the  routines  are 
not  recommended  for  general  purposes.) 

The  nature  of  the  output  of  these  routines  has  been 
given  In  detail  elsewhere . Briefly,  It  consists  in 
the  following  quantities  for  each  simplex  method  Iter¬ 
ation:  the  amount  of  Infeaslblllty ,  the  current  value  of 
the  objective,  thu  pivot  row  and  column,  the  determinant 
of  the  basis,  the  number  of  product-form  transformation 
entries,  the  number  of  arithmetic  operations  performed 
In  each  of  several  major  subdivisions  of  an  Iteration, 
and  the  number  of  non-zero  elements  In  certain  arrays  of 
Interest.*  At  the  end  of  a  problem,  the  complete  solu¬ 
tions  are  given  as  well  as  the  "errors" — the  extent  to 
which  the  final  solution  falls  of  being  both  primal  and 
dual  feasible.  All  solutions  obtained  have  been  checked 
with  those  obtained  by  other  routines  on  the  same  prob- 
lems,'  '  and  the  statistic-collecting  features  have  been 
checked  In  detail  for  most  of  the  runs  by  hand  calcu¬ 
lations  of  a  small  problem. 

* 

See  Sections  III  and  VIII  for  definitions  of  teirnis 
used  here. 
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The  experimental  data  are  organized  by  "runs,"  each 
of  which  consists  of  the  entire  set  of  test  problems,  by 
means  of  a  routine  embodying  a  particular  algorithm 
variation.  Of  the  runs  done  so  far  In  the  SCEMP  project, 
30  furnish  the  data  used  In  this  report;  the  others  bear 
on  matters  not  discussed  here.  Two  kinds  of  data  per¬ 
taining  to  a  run  have  been  used  In  this  report:  we 
consider  the  number  of  simplex  method  Iterations,  or 
changes  of  basis,  required  to  reach  a  certain  end--clther 
the  first  feasible  solution  or  the  optimal  solution  of 
the  problem  (see  Sections  IV-VIl);  and  we  discuss  the 
total  number  of  arithmetic  operations  required  (see 
Sections  VIII  and  IX).  The  Appendix  lists  the  raw  data 
from  which  the  figures  presented  In  the  sequel  have  been 
calculated. 

Since  the  point  of  most  of  these  experiments  has 
been  to  compare  alternative  methods,  the  following 
general  format  has  been  used  for  the  results.  The  ap¬ 
propriate  data  (e.g.,  number  of  Iterations)  for  a 
particular  run  are  chosen  as  a  base .  In  order  to  compare 
another  run  with  the  base,  the  datum  obtained  In  the  com¬ 
parison  run  for  each  of  the  test  problems  Is  divided  by 
the  corresponding  datum  for  the  base  run;  the  resulting 
ratio  Is  the  proportion  In  which  the  measure  has  been 
reduced  by  use  of  the  compared  procedure.  For  example, 
suppose  that  Algorithm  I  took  20  Iterations  to  solve 
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problem  ID  and  30  to  solve  problem  2A;  and  that  Algorithm 
II  took  l4  and  24  Iterations,  respectively.  Choosing 
Algorltlim  I  as  the  base,  the  comparative  results  would 
be  given  as  In  Table  1-1. 


Table  1-1 

Algorithm  II  Compared  with  Algorithm  I 


Problem 

ID 

2A 

avg. 

C  .V. 

Alg.  II 

.70 

0 

00 

.75 

.07 

Note  that  usually  the  average  of  the  ratios  Is 
given,  as  well  as  their  coefficient  of  variation  (the 
standard  deviation  divided  by  the  average).  The  prob¬ 
lems  will  be  listed  In  order  of  their  number  of  constraints. 
The  ratios  all  have  equal  weights  In  the  averaging,  but 
the  average  could  be  viewed  as  an  average  of  the  data  of 
the  compared  run  weighted  by  the  reciprocals  of  the  cor¬ 
responding  data  of  the  base  run.  For  this  reason,  the 
average  Is  a  somewhat  fairer  measure  when  the  data  of 
the  base  are  larger  than  those  of  the  compared  run. 

Owing  to  the  arithmetic  of  averaging.  If  II  were  chosen 
as  the  base  and  I  as  the  compared  run,  the  resulting 
average  would  be  greater  than  the  reciprocal  of  that  of 
1-1. 

Some  gaps  appear  In  the  tables  that  follow.  The 
largest  problem  cannot  be  run  on  the  routines  using  the 
standard  form  of  the  simplex  method,  and  two  other  prob¬ 
lems  were  omitted  from  some  "feasible  solution"  runs 


because  they  had  starting  feasible  solutions. 

A  good  deal  of  special  terminology  Is  used  In  de¬ 
scribing  the  computations.  Special  terms  are  usually 
defined  by  context  at  their  first  appearance,  which  Is 
signaled  In  underlining.  Most  of  them  are  Introduced 
In  Sections  III  and  VIII.  While  the  terms  "method"  and 
"procedure"  are  used  Interchangeably  In  a  very  general 
way,  we  use  the  term  algorithm  to  refer  to  any  particu¬ 
lar  version  of  the  simplex  method  which  chooses  the 
pivot  column  and  the  pivot  row  In  a  particular  manner, 
regardless  of  the  way  In  which  the  data  used  In  making 
the  choice  arj  obtained.  Thus  Sections  III  to  VII  study 
only  algorithms  and  the  data — principally  operation 
counts — associated  with  them,  while  the  remaining  sec¬ 
tions  study.  In  part,  diffcicnt  methods  of  performing 
the  same  algorithm. 
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II .  THE  PROBLIiMS 


The  linear  prograimr.ing  problems  on  which  cur  ex¬ 
periments  were  conducted  were  drawn  from  the  file  of 
thirteen  problems  maintained  by  the  Test  Problems  and 
Experiments  Committee  of  the  SHARE  Linear  Programming 
Project.  The  problems,  submitted  by  various  members  of 
the  Committee  In  1959  and  I960,  were  all  used  as  pro¬ 
duction  problems  in  their  buslnessesj  the  majority  arose 
in  oil  refining  studies.  None  were  especially  construc¬ 
ted  for  test  purposes,  or  thought  "pathological."  The 
original  problems  are  available  through  the  Committee, 

Four  of  the  problems  were  not  used  here.  Problem 
1C  Is  too  small,  4a  Is  too  large,  and  3A  and  3B  had 
awlcward  Input  features.  Thus  our  work  was  done  with  the 
nine  problems  of  Table  2-1. 

Throughout  this  report  the  problems  are  listed  In 
the  order  of  their  numbers  of  constraints.  In  Table  2-1, 
the  "name"  Identifies  a  problem  in  the  Committee's  files. 
All  the  problems  are  formulated  as  problems  of  minimizing 
a  linear  objective  function  unaer  linear  equality  con¬ 
straints;  some  problems  have  several  alternative  objective 
functions,  so  that  there  are  always  one  or  more  "addi¬ 
tional  rows."  In  our  runs  the  highest -numbered  objective 
row  was  used  and  the  remainder  Ignored. 

The  "number  of  variables"  Includes  all  the  vari¬ 
ables  of  the  problem  but  no  "artificial"  variables.  The 
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" number  of  entries"  Is  the  number  of  non-zero  quantities 
appearing  among  the  constraints  and  objectives.  Some 
further  data  regarding  starting  bases  for  the  problems 
are  given  In  Sec.  IV. 

Table  2-1 


The  Test  Problems 


Name 

ID 

2A 

IE 

lA 

5A 

IG 

IP 

2B 

IB 

Number  of  con¬ 
straints:  M 

27 

30 

31 

33 

34 

48 

66 

96 

117 

Number  of  ad¬ 
ditional 
rows :  K 

1 

2 

8 

1 

1 

1 

1 

3 

1 

Number  of  vari¬ 
ables 

^5 

103 

106 

64 

78 

102 

135 

162 

253 

Number  of 
entries 

252 

811 

855 

245 

391 

462 

644 

897 

1210 
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III ■  TERMINOLOGY 

In  order  to  describe  the  algorithms  studied,  we 
develop  here  some  of  the  terminology  connected  with  the 
simplex  method.  It  Is  not  Intended  to  discuss  the  pro¬ 
cedure  Itself,  which  Is  done  In  many  standard  works. 

The  discussion  In  this  section  Is  entirely  In  terms  of 
the  standard  form  of  the  simplex  method;  the  other  forms 
are  dealt  with  In  Sec.  VIII. 

Let  a  linear  programming  problem  have  N  variables 
Xj^,..,Xjj  and  M  equation  constraints.  At  any  stage  In 
the  simplex  method  solution  there  Is  defined  a  basis , 

which  Is  a  set  of  M  basic  variables,  say  x,  ,..,x.  ; 

Jl  Jj, 

let  the  remaining  variables  be  x.  ,..,x,  .  The 

^M+l  Jn 

current  tableau  is  the  set  of  coefficients  of  the  linear 
equations 

X.  'f'S-.-.X,  +  ***'hb.»uX,  “b.  , 

h  ^M+l  ^ 

+  ""mi  ...  ^,N-M  '’m  ' 

which  are  uniquely  defined  by  the  current  basis  and  the 

requirement  that  this  set  be  equivalent  to  the  linear 

equations  defining  the  original  problem.  We  say  that 

the  basic  variable  x,  occupies  position  1  In  the 
•^1 

basis,  for  1  =  1,..,M  . 


I 


It  l3  further  supposed  that  the  objective  function 
to  be  minimized  Is  expressed  at  this  time  In  terms  of 
the  nonbaslc  variables  as 


°N-M  ^J. 


the  coefficients  Cj  are  the  reduced  costs.  (The 
quantities  a^^j  and  Cj  defined  here  commonly  carry  a 
superior  bar  to  Indicate  that  they  change  In  each  iter¬ 
ation;  we  omit  the  bar.)  The  basic  solution  of  the 
equations  above  la  obtained  by  setting  all  nonbaslc 
variables  to  zero,  giving  the  basic  variables  the  values 
X,  =  >  *tc.,  and  the  objective  the  value  z  , 

In  a  single  iteration  of  the  simplex  method  a  pivot 
column  J  (where  Jj^^  Is  the  index  of  a  nonbaslc 
variable)  and  a  pivot  row  I  of  the  tableau  are  chosen, 
cind  the  roles  of  the  basic  variable  occupying  position 
I  and  the  nonbaslc  variable  associated  with  column  J 
are  Interchanged,  new  data  of  the  form  of  the  equations 
above  being  obtained  by  pivoting  on  the  entry  a^j  of 
the  tableau.  The  value  of  the  objective  changes  by  the 


amount 


h  /  «IJ 


Some  M  variables  must  be  chosen  as  the  starting 
basis  for  the  procedure.  If  not  all  these  variables 
belong  to  those  of  the  original  problem,  the  remainder 
are  artificial.  Each  Instance  of  a  basic  variable  whose 
current  value  Is  negative,  or  of  an  artificial  variable 


whose  value  Is  not  zero.  Is  an  Infeaslblllty .  A  basic 
solution  having  no  Infeaslbllltles  Is  feasible  ■  When  It 
la  necessary  to  adjoin  artificial  variables  In  order  to 
have  a  starting  basis,  we  always  adjoin  for  each  a  column 
of  coefflc'cnts  of  the  form  [O, . . ,1 , . . ,0l  to  the  original 
problem,  the  single  "l"  of  the  column  lying  In  a  row  cor¬ 
responding  to  an  otherwise  unoccupied  position  of  the 
basis.  When  there  are  Infeaslbllltles,  a  separate  ob¬ 
jective  function  Involving  them  Is  defined,  and  It  Is 
required  that  this  Infeaslblllty  objective  be  minimized. 
The  process  of  minimizing  that  objective  Is  Phase  One; 
the  subsequent  m'nlmlzatlon  of  the  proper  objective,  once 
a  feasible  solution  Is  obtained.  Is  Phase  Two . 

In  the  sequel  we  refer  to  the  ordinary  simplex  method, 
by  which  we  mean  the  simplex  method  as  most  commonly  pre¬ 
sented,  except  that  we  extend  the  usual  procedure  for  the 
choice  of  pivot  row  to  that  of  the  "composite  algorithm." 

In  Phase  Two  the  procedure  Is  quite  ordinary.  A 
pivot  column  J  Is  chosen  so  that  Cj  Is  minimal  (if  all 
are  non-nogatlve,  the  current  solution  Is  optimal).  Then 
the  pivot  row  I  Is  chosen  so  that  after  pivoting  the  cur¬ 
rent  solution  will  still  be  non-negative:  I  Is  the  1 
which  minimizes  b^^  /  a^j  for  all  a^j  >  0  .  If  bj  =  0 — 
degene;’acy--3hould  happen,  then  I  Is  chosen  as  the  1 
maximizing  a^^j  among  all  1  for  which  b^  =  0  .  (This 
rule  Is  not  known  to  prevent  "cycling,"  but  Is  very  ef¬ 
fective  In  practice. 
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In  Phase  One  the  objective  Is  defined  as  the  sura  of 

the  Infeaslbilitles:  5^  fx.  :  x,  <  0  or  x,  artificial]  . 

J  J  J 

The  reduced  cost  for  the  nonbaslc  variable  J  is  then 

E  :  b^  <  O]  -  E  f a^j  :  b^  >  0  and  position  1 

artificial}  . 

The  pivot  column  J  is  chosen  for  minimal  reduced  cost, 
and  the  pivot  row  I  so  that  no  variable  non-negative  in 
the  current  solution  becomes  negative  after  pivoting:  I 
is  the  1  which  achieves  the  smaller  of  the  two  ratios 

”l"l  f^AlJ  =  ‘^1  '  ^IJ  ^  f'^l/^lJ  '  ‘^l  '  ^IJ  <  ol 

A  somewhat  moi-e  complicated  rule  is  needed  for  degeneracy. 

In  the  absence  of  negative  b^^  ,  the  pivot-row  rule  oper¬ 
ates  Just  as  in  Phase  Two. 
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IV.  STARTING  BASES 

The  basis  with  which  a  problem  is  started  naturally 
has  a  great  iniluence  on  the  number  of  iterations  re¬ 
quired  to  solve  it.  In  practice  one  often  attempts  to 
guess  a  starting  basis  which  will  be  as  nearly  feasible 
and  optimal  as  possible;  a  sophisticated  routine  will  make 
good  use  of  such  a  guess  even  if  the  basis  is  Incomplete, 
infeasible,  or  singular.  The  three  methods  studied  here 
do  not,  of  course,  make  any  use  of  special  information 
about  the  problem;  they  assume  complete  ignorance,  and  may 
be  used  with  any  problem. 

N  basis;  When  no  starting  basis  is  specified,  a  full 
set  of  M  artificial  variables  is  adjoined  to  the  problem 
and  constitutes  the  starting  basis. 

3  basis;  By  singleton  we  mean  a  variable  having 
only  one  non-zero  entry,  and  that  positive,  In  the  equa¬ 
tions  of  the  initial  tableau.  An  S  basis  Is  a  starting 
basis  consisting  of  a  maximal  set  of  singletons,  with 
artificial  variables  used  as  necessary  for  the  unfilled 
positions.  (The  computational  cost  of  pivoting  on  single- 
tons  is  almost  nothing,  and  feasibility  Is  Improved  If  all 
the  original  right-hand  sides  are  non-negative.) 

F  basis;  A  full  basis  was  produced  by  this  procedure: 
first,  an  S  basis  was  chosen;  subsequently,  each  column 
of  the  tableau  was  examined,  and  pivoted  Into  the  basis  if 
it  had  a  non-zero  entry  corresponding  to  any  unfilled  basis 


position.  The  only  basis  positions  left  unfilled  by  this 
procedure  are  those  corresponding  to  redundant  constraints. 
Naturally,  the  resulting  basis  is  not  lllcely  to  be  primal 
or  dual  feasible.  Other  procedures  for  obtaining  a  full 
basis  have  been  tried  but  not  yet  fully  evaluated;  they  do 
not  seem  to  offer  much  advantage  over  the  above. 

Table  U-1  describes  the  bases  resulting  from  the  use 
of  procedures  S  and  F.  All  the  data  are  proportions,  the 
number  of  variables  in  a  given  category  being  divided  by 
the  number  of  constraints  in  the  problem.  The  last  two 
lines  constitute  the  proportion  of  Infeaslbllltles  in  the 
starting  basis.  Kote,  however,  that  artificial  variables 
initially  at  zero  level  tend  to  become  non-zero  before 
they  are  eliminated,  so  that  for  an  S  basis  the  total 
number  of  artificials  is  the  better  measure  of  Infeasl- 
blllty. 


Table  4-1 

Starting  Basis  Characteristics 


Problem 

ID 

2A 

IE 

lA 

5A 

10 

IP 

2B 

IB 

(S  basis) 

Singletons  used 

.19 

GO 

.19 

.30  1 

.00 

.90 

.65 

.86 

.29 

Positive  artificials 

.81 

.00 

.19 

.70 

.00 

.10 

.35 

.05 

.60 

Zero  artificials 

.00 

.13 

.61 

.00 

.00 

.00 

.00 

.08 

.11 

(P  basis) 


Negative  variables 


.41  .00  .42  .30  .00  .23  .26  .10  .46 
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Note  that  the  proportion  of  Infeaslbllltles  in  the 
F  baa';-,  rnna  a  little  more  than  half  the  proportion  In 
the  S  baa  la.  Wo  view  thla  ar,  accountlnf.  for  the  advan- 
tar.c,  to  be  aeon  below,  of  the  F  basis  over  the  3  basis. 

We  are  mainly  Interested  in  the  number  of  simplex 
method  Iterations  required  to  obtain  the  first  feasible 
solution  alter  the  starting  basis  has  been  constructed. 

(In  Sec.  IX  the  effect  of  the  work  required  to  produce 
the  starting  basis,  as  Included  In  the  total  work  to  solve 
the  problem.  Is  considered.)  Bases  N  and  S  have  been 
used  with  two  algorithms:  the  ordinary  procedure  and  the 
"ratio  pr-lclng"  procedure,  described  In  Sec.  VI.  The 
results  are  summarized  In  Table  4-2.  The  first  ll:ie  com¬ 
pares  basis  S  (run  21)  with  basis  N  (run  5»  used  as  the 
base),  for  the  ordinary  algorithm;  the  second  line  com¬ 
pares  basis  S  (run  6)  with  basis  N  (run  8,  used  as  base) 
for  the  ratio  pricing  algorithm.  The  ratios  thus  repre¬ 
sent  the  proportion  In  which  the  number  of  Iterations  In 
Phase  One  Is  decreased  by  using  an  S  basis  rather  than  an 
N  basis. 


Table  k~2 


3  Basis  Compared  to  N  Baiils  for  Two  Algorithms 


Problem 

ID 

2A 

IB 

lA 

5A 

10 

IF 

2B 

IB 

Avg. 

Ordinary  algorithm 

.89 

.00 

.69 

.61 

.00 

.51 

.42 

.83 

.50 

Ratio  pricing 

.86 

.01 

.85 

.82 

.00 

AO 

.58 

.83 

.54 
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Evldently  uae  of  an  S  baa  Is  entails,  on  the  average, 
a  saving  of  about  hQ%  In  the  number  of  Iterations  required 
for  Phase  One. 

Comparison  of  bases  S  and  P  has  been  made  In  each  of 
three  algorithms,  with  the  number  of  Iterations  for  basis 
S  taken  as  the  base  data:  the  ordinary  algorltiun  (runs 
21  and  39,  respectively);  the  sequential  procedure  (runs 
31  and  36);  and  the  leaat-lnfeaslblllty  procedure  (runs 
33  *nd  37)*  The  last  two  procedures  are  discussed  In 
Sec.  VI.  Table  4-3  sumiiarlzea  these,  omitting  problems 
2A  and  5A  because  their  starting  S  and  F  bases  are  feasible. 


Table  4-3 

P  Basis  Compared  to  S  Basis 


Problem 

ID 

IE 

lA 

IG 

IP 

2B 

IB 

Avg 

Ordinary  algorithm 

.32 

.71 

48 

1.62 

.05 

.69 

.53 

.63 

Sequential 

.37 

1.00 

■  31 

2.43 

.11 

1.07 

.55 

.84 

Least- Infeaslblllty 

.33 

.94 

.35 

2.27 

.09 

•93 

.78 

.81 

The  overall  average  of  these  proportions  Is  O.76,  pre¬ 
dicting  a  saving  of  24^  In  use  of  an  P  basis  rather  than 
an  S  basis. 

We  conclude  that  In  the  absence  of  other  knowledge  of 
the  problem,  an  F  basis  should  be  used.  Some  linear  pro- 
grammlng  routines'^'  make  It  possible  to  use  a  mixed 
procedure,  entering  a  known  partial  basis  and  subsequently 
completing  It  In  an  arbitrary  manner. 
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We  may  try  to  predict  the  number  of  Iterations  Phase 
One  requires  using  the  ordinary  algorithm.  Each  entry  In 
Table  4-4  Is  obtained  by  averaging,  for  all  problems,  the 
number  of  Iterations  taken  using  the  basis  N,  S,  or  P 
divided  by  one  of  three  possible  measures  of  problem  dif¬ 
ficulty — the  number,  M,  of  constraints,  the  number  of 
non-slngletons ,  or  the  number  of  Infeaslbllltles  In  the  P 
basis.  Thus,  for  example,  the  number  of  Iterations  re¬ 
quired  using  an  S  basis  Is  expected  to  be  .78  M.  The 
coefficients  of  variation  are  given  in  parentheses.  It 
Is  disappointing  that  the  number  of  constraints  Is  a  bet¬ 
ter  basis  for  prediction  than  the  more  Informative  measures. 

Table  4-4 

Phase  One  Iterations  vs.  Measures  of  Problem  Difficulty 

Measure 


M 

Number  of 
non-slngletons 

Number  of 
In  P 

negatives 

basis 

N 

1.69 

(.3) 

Starting 

Basis 

S 

.78 

(.8) 

2.13 

(.9) 

P 

.56 

(.6) 

2.07 

(1.1) 

2.12 

(.8) 
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V.  THE  FEASIBLE  SOLUTIOW 

In  general.  Phase  One,  the  task  of  obtaining  a  first 
feasible  solution.  Is  accomplished  by  employing  the  simplex 
method  to  minimize  some  measure  of  the  Infeaslblllty  of  a 
solution.  The  five  procedures  studied  here  employ  four 
different  measures  of  infeaslblllty.  In  all  of  them  the 
measure  constitutes  an  objective  function  whose  reduced 
costs  are  calculated  so  that  the  choice  of  pivot  column 
can  be  made  by  the  ordinary  rule.  In  all  but  the  "ex¬ 
tended  composite"  algorithm  the  ordinary  rule  of  pivot  row 
selection  Is  used. 

The  ordinary  procedure  Is  described  In  Sec.  III. 

The  extended  composite  procedure^ differs  from 
the  ordinary  In  choice  of  pivot  row.  After  the  pivot 
column  has  been  chosen  In  the  ordinary  way,  the  pivot  row 
Is  selected  so  that  the  sum  of  Infeaslbllltles  after 
pivoting  will  be  minimized;  variables  are  allowed  to  change 
sign  freely.  Thus  I  Is  defined  by  9^  -  bj/Sjj  yhgpg 

9jj  minimizes  -  6  :  b^^  -  9  a^^j  Is  Infeasible} 

In  the  sequential  procedure,  the  Infeaslblllty  Is 

corrected  one  component  at  a  time.  In  order.  At  any 

Iteration,  let  1^  be  the  least  1  for  which  some  b^  • 

Is  Infeasible,  and  the  corresponding  variable.  Tlie 

objective  for  minimization  Is  defined  as  x^.  If  position 

Ijj  Is  artificial  and  bj^  Is  positive,  or  as  -x  If 

■*"0 

b.  Is  negative.  (T5ie  reduced  cost  for  column  >  will 
o 
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then  be  Just  a,  ,  or  -a.  ,  . )  IXjrlng  the  procedure, 
row  Ijj  will  be  made  feasible,  feasibility  on  the  previous 
rows  being  preserved . 

The  least-lnfeaslbillty  procedure  is  like  the  sequen¬ 
tial,  except  that  at  each  iteration  the  index  1^  is 
taken  so  that  x^,  is  minimal  among  all  Infesuslble  vari¬ 
ables;  the  Index  may  Increase  or  decrease. 

The  fudge  procedure,  but  not  its  name,  is  due  to  Oass.''^  ' 
A  problem  having  negative  solution  values  is  augmented  by 
a  single  ai-tlflcli  I  variable  and  subjected  to  a  trans¬ 
formation  yielding  non-negative  solutions  for  the  aupnented 
problem.  .Specifically,  the  tableau  is  augmented  by  a 
column  containing  the  entry  -1  in  each  row  having 
b^<  0  and  zeros  elsewhere;  and  the  desired  tableau  is  ob¬ 
tained  by  pivoting  in  the  1^^  entry  of  the  added  column, 
wliere  bj  =  mln^  b^  .  Subsequently  the  sum  of  all  the 
artificial  variables  is  minimized  using  the  ordinary 
algorithm;  when  It  hais  been  reduced  to  zero,  a  feasible 
solution  Is  at  hand.  (Of  course,  other  means  of  getting 
feasible  could  be  used  once  the  negativity  has  been  re¬ 
moved.  ) 

Table  5-1  lists  the  runs  done  using  these  five  pro¬ 
cedures,  indicated  at  the  left.  The  starting  basis  used 
is  listed  at  the  top. 

*Pp.  120-125. 
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Table  5-1 

Feasible  Solution  Runs 


N 

S 

F 

Ordinary  algorithm 

5 

21 

39 

Extended  composite  algorit.im 

(5) 

(21) 

38 

Sequential  algorithm 

31 

36 

Least-lnfeaslblllty  algorithm 

33 

37 

Pudge  procedure 

(5) 

(21) 

32 

Runs  Indicated  In  parenthetes 

were 

not  done. 

since 

the  sane  results  would  have  been  obtained  as  In  the  run 
whose  number  Is  given. 

Tables  5-c  and  5-3  give  the  results  for  these  pro¬ 
cedures,  for  bases  S  and  P,  relative  to  the  ordinary 
procedure.  The  last  line  of  each  table  Is  the  proportion 
of  Infeaslbllltles  In  the  starting  basis  for  each  problem. 

Table  5-2 


Phase  One,  S  Basis,  Relative  to  Ordinary  Algorithm 


Problem 

ID 

IE 

lA 

10 

IF 

2B 

IB 

Avg. 

Sequentlea 

.96 

.94 

1.52 

1.00 

.98 

.97 

1.04 

1.06 

Least-lnfeasl- 

blllty 

1.08 

1.03 

1.36 

1.05 

1.24 

1.12 

.94 

1.12 

Proportion  Infeasl¬ 
blllty 

.81 

.19 

.70 

.10 

.35 

.05 

.60 

The  eunount  of  Infeaslblllty  does  not  seem  to  affect 
the  relative  efficiencies  of  these  methods  much,  although 
It  does  affect  the  totrl  work  done,  as  the  data  for  runs 
21,  31,  and  37  In  the  Appendix,  or  those  of  Table  4-4,  show. 
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T»bl«  5-3 


Fhu«  OiM,  F  Basla,  Relatlva  to  Ordinary  AlRorltha 


Problem 

ID 

It 

lA 

10 

IF 

2B 

IB 

Avg. 

Extended  oonpoeite 

1.00 

1.06 

1.A2 

.65 

1.00 

l.OA 

1.01 

1.03 

Sequential 

1.12 

1.33 

1.17 

1.50 

2.00 

1.51 

1.06 

1.39 

Laeet-lnfaaal- 

blllty 

1.12 

1.37 

1.00 

l.*7 

2.00 

1.51 

1.39 

1.41 

Fudae 

1.25 

1.06 

1.25 

1.00 

2.00 

1.3* 

1.01 

1.27 

Proportion  Infaael- 
blllty 

.«! 

.t2 

.30 

.23 

.26 

.10 

.46 

The  results  pretty  well  establish  the  ordinary  pro¬ 
cedure  superior  In  getting  feasible.  Its  objective  Is 
responsible,  since  the  mlnlalzation  algorithm  Is  the  same 
In  all  the  runs.  It  seems  that  by  moving  In  a  direction 
tending  to  minimize  the  sum  uf  all  the  Infeaslbllltles  we 
give  more  chance  to  a  number  of  Infeaslbllltles  to  leave, 
while  the  sequential  and  least-feasible  procedures,  con¬ 
centrating  on  a  single  variable  at  a  time,  are  too  single- 
minded.  Since  several  negative  infeaslbllltles  can  be 
removed  In  one  Iteration,  while  only  one  artificial  variable 
can,  it  Is  reasonable  that  the  difference  Is  more  decisive 
for  F  bases  than  for  S  bases. 

The  extended  composite  procedure  Is  somewhat  disap¬ 
pointing.  It  might  work  better  If,  at  the  expense  of 
considerably  more  calculation,  the  pivot  column  were  chosen 
by  the  same  criterion  as  Is  the  pivot  row. 
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VI.  THE  OPTIMAL  SOLUTION 


Of  greatest  Interest  to  the  ordinary  user  is  the  amount 
of  work  required  to  solve  a  complete  problem.  In  this 
section  six  algorithms  are  compared  in  the  number  of  Iter¬ 
ations  required  to  obtain  an  optimal  solution.  All  but 
one  of  these  are  designed  to  handle  artificial  variables; 
for  them,  the  ordinary  Phase  One  objective — the  sum  of  all 
Infeaslbllltles — Is  used;  this  was  found  most  efficient 
In  Sec.  V.  Unless  otherwise  noted,  each  procedure  uses 
the  same  method  for  minimizing  Its  objective  In  Phase  iMe 
as  It  does  In  Phase  Two,  only  the  definition  of  the  ob¬ 
jective  changing  between  the  phases.  Similarly,  each 
procedure  (except  the  "symmetric")  uses  the  ordinary  choice 
of  pivot  row.  They  differ  primarily  In  the  manner  of 
choosing  the  pivot  column. 

The  ordinary  procedure  was  described  In  Sec.  III. 

The  posltlve-nonnallzed  procedures  (PNl  and  PN2)  can 
be  viewed  as  representative  of  those  proposals  which  aim 
at  eliminating  the  effects  of  bad  scaling  of  the  problem 
data  by  dividing  the  reduced  costs,  used  In  choosing  the 
pivot  column,  by  some  combination  of  the  coefficients 


a^^j  .  The  first  of  the  two  considered  here,  proposed  by 

Dickson  and  Frederick,^ uses  the  formula  d.  - 
2  2+^ 

°J  ^°J  ^1  ®1J  ),  where  a^j  Is  the  "positive  part"  of 

,  choosing  the  pivot  column  as  that  J  for  which  dj 


Is  maximal  for  Cj  <  0  .  Tlie  procedure  PN2  Is  essentially 
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2  +  *” 

this,  uslny  Instead  the  formula  dj  =  Cj  /  j  ,  which 

elves  the  same  result. 

The  PNl  procedure  employs  the  slightly  simpler  formula 

d  ,  =  0 ,  /  r.  a|,  ,  with  the  pivot  column  chosen  for  mini- 
J  J  X  1 J 

itial  dj  . 

Tlie  greatest-change  procedure  was  described  long  ago,^^^ 
but  has  been  little  used.  That  column  Is  chosen  which, 
after  pivoting,  will  give  the  greatest  decrease  In  the 
value  of  the  objective;  It  la  the  J  which  minimizes  the 
expression  Cj  mlnj^fb^^/a^^ j  :  a^^j  >  0)  for  the  change  of 
the  objective. 

The  ratio  r '‘Icing  procedure  was  suggested  Informally 
by  Markowitz  some  time  ago.  It  differs  from  the  ordinary 
procedure  only  In  Phase  One.  Letting  Wj  be  the  reduced 
cost  for  the  Infeaaibllity  objective  then,  and  Cj  be 
the  reduced  cost  for  the  proper  objective,  the  pivot 
column  J  Is  chosen  so  as  to  maximize  ®  ‘ 

we  obtain  the  largest  possible  Improvement  In  the  proper 
objective  per  unit  change  of  Infeaslblllty .  It  may  be 
viewed  as  an  application  of  parametric  linear  programming:''^' 
defining  ji*  at  each  Iteration  as  the  largest  0  such 
that  Cj  +  Wj  >  0  for  all  Wj  <  0  ,  the  pivot  column 
Is  chosen  so  as  to  Increase  )S*  .  Evidently  when  f!* 
becomes  sufficiently  large  we  have  all  Wj  >  0  ,  and  Phase 
One  Is  ended.  It  turns  out  that  almost  all  Cj  eu’e  then 
non-negative,  too,  so  that  Phase  Two  Is  quite  short.  The 
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alm  of  the  procedure  Is  to  obtain  a  first  feasible  solu¬ 
tion  which  Is  nearly  optimal;  the  data  of  the  Appendix 
for  run  6  show  that  It  does  this  well. 

( 12) 

The  symmetric  procedure  of  Talacko'  '  Is  employed 
only  with  a  full  basis;  It  may  take  either  "primal"  or 
"dual"  simplex  method  steps.  For  one  Iteration:  Among 
those  columns  with  negative  reduced  costs  and  those  rows 
whose  basic  variables  are  non-negative,  a  potential  pivot 
Is  determined  using  the  greatest-change  procedure  as  de¬ 
scribed  above;  and  among  columns  with  positive  reduced 
costs  and  rows  with  negative  variables,  a  potential  pivot 
Is  determined  using  the  dual  of  the  greatest-change  pro¬ 
cedure  (for  which  the  greatest  increase  of  the  objective 
Is  sought).  That  pivot  Is  used  for  which  the  magnitude 
of  the  objective  change  Is  greater.  If  a  step  of  the  first 
kind  Is  taken,  all  non-negative  basic  variables  stay  non- 

negative.  The  procedure  does  not  always  terminate  In  a 

( 12*) 

solution  of  the  problem;'  '  but  It  did  for  the  test 
problems . 

The  parametric  procedure  of  Dantzlg^^^^  Is  likewise 
employed  conveniently  only  with  a  full  basis,  and  takes 
either  primal  or  dual  steps.  It  Is  generally  like  the 
symmetric  procedure,  but  Its  termination  Is  known.  At 
the  beginning  of  procedure  a  column  of  right-hand  side 

*P.  10. 


changes  1e  added  to  e  tableau  so  ttiPt  vtxri.  the  column 
Is  multiplied  by  an  Initial  value  *  0)  ol  the  param¬ 
eter  ^  and  added  to  the  current  hnnlo  «/»lutlon  the 
result  will  be  non-neg-atlve;  and  Llkewiee  a  row  of  cost 
changes  Is  added  to  the  tableau  so  that  when  multiplied 
by  the  Initial  value  of  and  added  to  the  current  re¬ 
duced  costs,  the  result  will  be  non-negative.  In  this 
manner  a  problem  Involving  the  single  parameter  ^  has 
been  defined  which  Is  both  primal  and  dual  feasible  when 
ji  =  and  Is  equivalent  to  the  original  problem  when 
jS  =  0  .  The  procedure  consists  in  then  reducing  0 
using  the  pivot  choice  rules  of  the  parametric  linear 
programming  algorithms , which  will  give  a  sequence 
of  primal  and  dual  feasible  solutions  for  a  decreasing 
sequence  of  values  of  ft  terminating  In  zero. 

Table  6-1  compares  all  these,  taking  the  ordinary 
procedure  as  the  base  (run  21 ). 

Of  the  runs  with  singleton  basis,  the  positive- 
normalized  procedures  are  outstanding,  and  overall  the 
great eat -change  procedure  with  full  basis  Is  best.  Un¬ 
fortunately  the  positive-normalized  procedures  .cave  not 
yet  been  tried  with  full  bases;  they  might  perform  even 
better.  Incidentally,  the  data  of  the  Appendix  show  that, 
with  the  natural  exception  of  ratio  pricing,  the  dif¬ 
ferences  among  the  procedures  are  reflected  In  Phase  One 
In  about  tlie  same  way  as  In  the  entire  process. 
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(Note:  If  problem  IB  is  eliminated  from  39  and  41,  the  averages  are  .84,  .69.) 
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The  data  of  Table  6-1  allow  the  symmetric  and 
greatest -chance  procedures  to  be  compared  directly  with 
the  ordinary  procedure  with  full  basis.  Using  run  39  as 
base,  the  averages  and  coefficients  of  variation  obtained 
are:  symmetric  algorithm,  .92,  .3;  greatest-change 
algorithm,  .78,  .3;  parametric  algorithm,  1.25,  .2.  The 
relative  efficiencies  of  these  procedures  are  not  changed 
much  by  calculating  them  from  the  different  base  run. 

It  la  of  considerable  Interest  to  find  some  means  of 
predicting  the  work  needed  for  a  problem  about  which 
little  Is  known.  In  Table  4-4  It  was  found  that  the 
number  M  of  constraints  was  the  best  guide  of  those 
studied  to  the  number  or  Iterations  for  Phase  One;  we  shall 
use  It  also  In  connection  with  the  total  iterations  re¬ 
quired.  Table  6-2  thus  lists  the  number  of  Iterations 
required  to  solve  each  of  the  problems  using  the  ordlnarj’ 
algorithm  divided  by  M  .  It  would  appear  that  rule  of 
"2M  Iterations"  from  folklore  Is  fairly  good  when  a 
singleton  basis  is  used. 

Table  6-2 

Iterations/Constraints  for  Ordinary  S  basis  (run  21) 

ID  2A  IE  lA  5A  IG  IF  2B  IB  avg.  c.v. 

2.00  1.67  1.71  1.27  1.09  1.29  1.83  1.18  3.33  1-71  .4 

The  corresponding  data  for  the  algorithms  of  Table 
6-1  can  be  found  by  multiplying  the  entries  of  6-2  by 
those  of  6-1.  The  averages  thus  obtained  appear  In  Table 

6-3. 


Table  6-3 


Suimnary  of  Iteration:.,  ts 


Algorithm 

Run 

Singleton  basis 

Avei’ay 

Ordinary 

21 

1.71 

PNl 

10 

1.24 

PN2 

14 

1.24 

Greatest-change 

15 

1.36 

Ratio  pricing 

6 

1.50 

Pull  basis 


Ordinary 

39 

1.39 

Symmetric 

40 

1.13 

Or  e  at  e  3 1  -  c  hange 

41 

•98 

Parametric 

59 

1.60 

A  more  detailed  examination  of  the  data  seems  to 
show  that  the  dependence  of  the  number  of  Iterations  on 
M  could  be  better  expressed  by  a  formula  of  the  form 
k  m'’  ,  where  b  Is  slightly  leas  than  one,  but  this  Is 

not  clear.  Using  a  singleton  basis  an  estimate  of  between 
M  and  3M'  Iterations  will  almost  always  be  correct. 


^  c\j  cj  .St  m  <\j  ro 
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VII.  SUBOPTIMIZATION 


Versions  of  suboptimization  have  been  used  for  some 
time  In  linear  pro^jrammlng  routines  bothered  by  small 
core  size,  but  the  advantages  of  a  version  of  It  for  routines 
for  which  core  size  Is  no  particular  heuidlcap  were  first 
exploited  by  D.  M.  Smith. As  used  here,  the  course 
of  the  solution  of  a  problem  consists  of  a  number  of 
passes .  at  the  beginning  of  each  of  which  some  number  L 
of  nonbaslc  columns  Is  selected  ais  a  set  of  candidates  for 
pivoting  (those  having  the  L  minimal  reduced  costs  are 
chosen).  During  th-c  pass  no  other  nonbaslc  columns  are 
considered;  simplex  method  Iterations  are  performed  using 
the  selected  columns  until  the  objective  has  been  mlnljttl.ted 
on  that  subset.  (A  basic  coliuT.n  which  becomes  nonbaslc 
during  the  pass  is  not  further  considered.) 

Tlie  number  L  of  candidates  Is  an  Important  para> 
meter;  values  of  2,  3,  s.  and  0  were  used  here.  During  a 
pass,  any  of  the  various  means  of  selecting  a  pivot  coliann 
discussed  previously  might  be  used  In  minimizing  the  ob¬ 
jective  on  the  candidates.  Three  were  tried  here:  the 
ordinary  procedure  of  minimal  reduced  cost;  the  greatest- 
change  procedure:  and  the  procedure  PNl. 

Both  the  number  of  Iterations  and  the  number  of  passes 
required  to  solve  a  problem  are  of  Interest.  In  the 
table  below,  the  numbers  required  are  all  compared  with 
the  number  of  Iterations  used  by  the  ordinary  simplex  method 
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(run  21),  which  would  be  the  number  of  passes  for  any  of 
the  algorithms  for  L  =  1  .  Only  the  averages  and  coefficients 
of  variation  are  given  for  those  runs;  the  Individual  data 
fluctuate  considerably  less  than  In  most  of  our  experi¬ 
ments  .  An  Interesting  feature  of  the  raw  data  not  re¬ 
flected  in  the  averages  is  that  the  greatest-change 
procedure  commonly  requires  fewer  Iterations  under  sub- 
optimization  than  does  the  ordinary  procedure  vrlthout  it, 
which  is  generally  not  the  case  for  the  other  methods . 

Table  7-1 

Suboptimization  Runs  Compared  to  Ordinary  Algorithm 


Run 

Algorithm 

L 

Iterations 
average  c 

.V. 

Passes 

average  c . v 

ordinary 

2 

1.15 

.2 

.72  .2 

2? 

It 

3 

1.28 

.2 

.60  .2 

2Q 

It 

5 

1.26 

.2 

.45  .2 

29 

II 

8 

1.31 

.3 

.37  .4 

2? 

greatest- 

2 

1.07 

.2 

.72  .2 

24 

change 

3 

1.08 

.2 

.59  .2 

25 

It 

5 

1.08 

.3 

.45  .3 

26 

II 

8 

1.13 

.3 

.40  -3 

42 

PNl 

2 

1.15 

.1 

.72  .2 

43 

II 

3 

1.22 

.2 

.58  .2 

The  term 

"pass 

"  arises  from 

the 

fact  that  It  is  only 

necessary  to  consult  the  data  for  the  entire  problem  once 
during  a  pass;  the  data  which  have  to  be  retained  for  the 
subsequent  suboptimization  are  much  fewer.  This  fact 
makes  it  particularly  valuable  in  product  form  routines 


cou)  ro  ro  -P=^i\>ruro 
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and  those  v^hlch  use  tapes  extensively.  (The  three  main 
forms  of  the  simplex  .method  are  discussed  In  Sec.  VIII.) 
The  significance  of  the  statistics  above  depends  on  the 
for-m  of  routine  used.  In  the  product  form,  the  total 
work  done  depends  largely  on  the  number  of  passes;  In  the 
standard  form,  on  the  number  of  Iterations;  and  the  ex¬ 
plicit  form  Is  Intermediate.  Thus  suboptimization  is  of 
value  In  the  product  form  even  for  all-ln-core  routines, 
but  not  In  the  standard  form.  Three  production  linear 
programming  routines  now  use  It  In  the  manner  described. 

They  are  all  product  form  routines,  one  using  the  ordinary 

(q\ 

algorithm  with  L  »  2,^'  another  the  greatest-change 
algorltlun  v/lth  L  ■  and  the  third  has  options  for 

either  jilgorlthm  and  any  L  up  to 
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VIII.  OPERATIONS  AfJD  FORMS 

So  far  we  have  been  concerned  only  v/lth  the  number 
of  Iteratlonr  required  to  solve  a  problem.  A  better 
guide  to  the  computational  efficiency  of  a  procedure  Is 
the  number  of  floating-point  arithmetic  operations  per¬ 
formed — the  work  which  must  be  done  no  matter  hov;  the 
algorithm  Is  Implemented.  While  logic  and  bookkeeping 
time  are  usually  appreciable,  and  vary  between  different 
algorithms  and  different  forms  of  the  simplex  method.  It 
Is  precisely  In  such  non-arlthmetlc  work  that  computers 
and  programming  systems  differ  the  most.  Having  program¬ 
med  each  of  the  procedures  studied  here  as  economically 
as  we  could  from  the  standpoint  of  arithmetic,  we  feel  that 
the  results  on  arithmetic  work  come  close  to  a  machine- 
independent  measure  of  efficiency. 

Although  it  would  be  possible  to  count  separately 
each  elementary  operation,  it  turns  out  that  there  are 
only  three  combinations  of  elementary  floating-point 
operations  used  significantly  often  In  each  of  the  major 
subdivisions  of  an  iteration:  addition  and  multiplication; 
division  and  subtraction;  and  addition  alone.  Eacli  of 
the  following  three  .groups  Is  thus  called  one  operation: 

1  floating  add  and  1  floating  multiply  (l7.f|) 

1  floating  divide  and  1  floating  add  ( 19.^0 

3  floating  adds  (19.2). 
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The  average  number  of  7090  cycles  taken  by  each  combi¬ 
nation  Is  given  In  parentheses .  Wiille  some  error  Is  made 
In  considering  all  these  equivalent,  It  la  very  small, 
because  the  first  combination  accounts  for  almost  all  the 
calculations.  In  all  cases  (except  for  a  portion  of  the 
reverse-transformation  calculation  In  the  product  foimi) 
an  operation  Is  counted  only  when  both  operands  are  non¬ 
zero  . 

Tliere  are  many  ways  of  calculating  the  data  required 
for  the  steps  of  the  simplex  method.  In  all  of  them  the 
data  used  in  the  ordinary  procedure  are  obtained,  but  In 
different  ways.  The  three  main  forms  of  the  method  are 
described  below  In  outline;  the  details  may  be  found  In 
the  literature. In  considering  the  number  of  opera¬ 
tions  performed  In  one  Iteration  In  any  form,  it  Is  con¬ 
venient  to  have  a  priori  estimates  In  terms  of  M  (the 
number  of  constraints),  N  (the  current  number  of  variables), 
and  M  +  K  (the  total  number  of  rows  of  data).  In  the 
formulas  below,  factors  of  proportionality  e  between 
zero  and  one  reflect  the  fact  that  operations  Involving 
zero  data  are  not  counted;  and  quantities  of  order  smaller 
than  V?  are  disregarded. 

The  standard  form  Is  done  Just  as  the  ordinary  pro¬ 
cedure  Is  described  In  Sec.  III.  Pivoting  In  the  tableau 
Is  most  of  the  work  (requiring  ej^(M+K)  (N-M)  operations). 

Botii  forms  of  the  "revised  simplex  method"  calculate 
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needed  Items  of  the  tableau  by  multiplying  parts  of  the 
original  matrix  A  by  parts  of  the  inverse .  the  Inverse 
of  the  M  +  K-order  matrix  consisting  of  the  basic 
columns  of  A  .  Tlie  reduced  costs  are  obtained  by 
multiplying  A  by  the  prices .  that  row  of  the  Inverse 
corresponding  to  the  objective  row  of  A  +  K)  (N-M) 

operations ) ;  the  selected  pivot  column  of  the  tableau  is 
obtained  by  multiplying  the  appropriate  column  of  A  by 
the  Inverse  (the  number  of  operations  required  for  this 
and  the  remaining  steps  differs  for  the  two  forms);  the 
pivot  row  is  selected  as  usual;  and  pivoting  is  done  both 
in  the  Inverse  and  the  current  solution. 

In  the  explicit  form,  or  the  "revised  simplex  method 
with  explicit  form  of  the  Inverse,"  the  inverse  is  a 
square  M  +  K-order  matrix,  all  of  which  is  pivoted  in 

p 

at  each  Iteration.  Pivoting  requires  6jj(M  +  K) 
operations,  and  the  prior  multiplication  for  the  pivot 

p 

column  requires  +  K)  operations. 

In  the  product  form,  or  the  "revised  simplex  method 
with  product  form  of  the  inverse,"  the  inverse  is  main¬ 
tained  as  a  sequence  of  transformations,  each  of  which, 
having  at  most  M  +  K  nonzero  entries,  constitutes  the 
nontrivial  portion  of  the  pivot  column  of  the  tableau  as 
of  some  previous  Iteration.  Applied  appropriately,  these 
transformations  accomplish  the  work  of  matrix  multipli¬ 
cation  required  by  the  revised  simplex  method.  In  a 
pivot  step  these  data  are  not  altered  but  are  au^ented 
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by  one  rnot'o  transformation.  Their  total  number  is 

p 

j'jOnerally  somewhat  less  than  (M  +  K)  ,  and  most,  but 
not  all,  of  tnem  are  used  once  in  obtaining  the  prices 
(6^(K  +  operations)  and  the  pivot  column  (9g(M  +  K) 
operations).  The  number  of  accumulated  transformations 
is  periodically  reduced  by  "relnver-^lon,"  the  reconstruc¬ 
tion  of  a  produet-form  Inverse  from  A  in  a  minimal 
sequence  of  pivots.  The  routines  used  here  relnvert  auto¬ 
matically  at  those  points  they  determine  will  minimize  the 
total  operation  count  for  the  calculation. 

In  summary,  the  formulas  of  Table  8-1  indicate  the 
dependence  of  the  number  of  operations  per  iteration  on 
problem  size. 

Table  8-1 

Number  of  Operations  per  Iteration 
Standard  form  9^^  (M  +  K)  (N  -  M) 

Explicit  form  (N  +  K)  (N  -  M)  +  6^  (M  +  K)^  +  (M  +  K)^ 

Product  form  Og  (M  +  K)  (N  -  M)  +  9^  (M  +  K)^  +  Bg  (M  +  K)^ 

It  is  beyond  the  scope  of  this  study  to  discuss  the 
factors  9  of  these  formulas  in  detail.  They  will  be 
used  Instead  as  guides  to  the  scaling  of  our  operation 
counts.  Since  for  our  problems  N  is  closely  propor¬ 
tional  to  M  (n/M  ranges  from  1.67  to  3-43,  averaging 
2.31  with  coefficient  of  variation  O.27),  each  of  the 


-3*"- 


formulas  has,  approximately,  (M  +  K)  as  a  commcn 
factor.  Thus,  comparative  data  for  the  three  forms  can 
be  obtained  as  follows:  for  each  problem,  divide  the 
total  number  of  operations  required  to  solve  It  by  the 

O 

number  of  Iterations,  and  divide  the  result  by  (M  +  K) 
The  data  of  Table  8-2  were  obtained  In  that  way;  for  all 
forms,  the  ordinary  simplex  algorithm  was  used,  and  an 
S  basis. 


Table  8-2 

Operattona  per  Iteration  /  (Me  K)^ 


Problam 

ID 

2A 

IE  lA 

5A 

10 

IF 

2B 

IB 

avg. 

c.v 

Standard  fonn 
(run  12) 

.68 

3.00 

2.11  .44 

1.28 

.79 

.76 

.59 

1.23 

.7 

Explicit  form 
(run  21) 

■  71 

.95 

.77  .49 

.5* 

•  31 

.38 

.29 

.67 

•  57 

.4 

Product  form 

t _  \ 

.56 

1.00 

.5*  .32 

.*5 

.27 

.25 

.1* 

.26 

.42 

.4 

(run  56) 


The  decrease  of  the  ratios  with  size  of  problem  Is 
noteworthy;  It  Is  probably  due  to  the  decrease  of  the 
proportion  of  nonzero  matrix  entries.  Table  8-3  makes  a 
more  direct  comparison  of  t)iese  data,  using  the  explicit 
form  run  as  a  base.  Note  that  the  relative  efficiency 
of  the  product  form  tends  to  Increase  with  the  size  of 
problem,  owing,  we  think,  to  Its  greater  ability  to  take 
advantage  of  the  lower  density  of  nonzeros. 

Tabla  8-3 

Standard  and  froduet  Compartd  with  Explicit  Porr 
ProblMi  ID  2A  IE  lA  ^A  10  IP  SB  IB  avg.  c.v. 

Standard  form  l.?8  A.67  3-03  .83  2.30  2.57  2.04  2.63  2.42  .  .4 

.82  1.05  .7»  .6*  .79  .86  .67  .li5  .»3  -71  .3 


Product  fonii 
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IX.  ALGORITHMS  COMPARED  BY  OPERATIONS 

The  algorithms  of  Sec.  VI  may  fln.’illy  be  compared  In 
the  total  number  of  operations  they  require  to  solve  a 
problem.  In  Table  9-1  they  are  all  compared  with  the 
ordinary  algorithm  In  explicit  form  (run  21).  With  the 
exception  of  that  procedure,  each  algorithm  given  has 
been  run  In  that  form  of  the  simplex  method  best  suited  to 
It;  the  ratio  pricing  and  the  greatest-change  (with  P 
basis)  procedures  are  omitted  because  they  were  not. 


T»ble  9-1 


Various  Algorithns  Compared  with  Ordinary,  Explicit 


ProMem 

ID 

2A  IE 

lA 

5A 

10 

IP 

2B 

IB 

(Vg. 

c.v 

Singleton  bull 

Pia- '•standard 
run  10 

.75 

2.1>l  2.E2 

.67  1.54 

1.21 

1.3?  1.37 

l.»3 

.4 

PN2--standard 
run  It 

1.07 

2.53 

.53 

1.79 

1.52 

1.18 

1.82 

1.66 

.4 

Oreatest'Change 
standard  run  15 

1.15 

3.66  3.72 

33 

2.3T 

.99 

1.66 

2.36 

2.03 

.6 

Ordinary* • produe t 
run  5o 

.82 

1.05  .7* 

.64 

.79 

.86 

.67 

.45 

•  *3 

.71 

•  3 

Pull  basis 

SynsMtrle  stand¬ 
ard  run  to 

.57 

4.80  6.04 

.47  2.81 

2.64 

1.01 

1.42 

2.47 

.8 

Ordinary -- produc t 

.63 

1.48  .81 

.tt 

.89 

.86 

.25 

.37 

.38 

.68 

.5 

run  55 


We  think  that  these  figures  constitute  the  best 
overall  assessment  of  these  alternative  algorithms  from 
the  point  of  view  of  calculation  needed.  The  product  fom 
of  the  ordinary  algorithm  seems  definitely  superior,  with 
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use  of  a  full  basis  probably  being  worthwhile  for  the 
larger  problems . 

We  may  try  to  predict  the  operation  count  for  an 
unknown  problem  of  given  size.  In  Table  9-2,  the  counts 
of  run  21  have  been  scaled  In  a  manner  Intended  to  elimi¬ 
nate  moat  of  the  Influence  of  the  size  of  the  problem. 

p 

Using  the  factor  (M  +  K)  as  In  Table  8-2  to  scale  the 
count  per  Iteration,  and  the  factor  M  as  In  Table  6-3 
to  scale  the  number  of  Iterations,  we  obtain  the  quotients 
of  9-2.  The  corresponding  quotients  for  the  other  runs 
can  be  obtal:  ed  by  multiplying  those  of  9-1  by  these 
numbers;  the  averages  and  coefficients  of  variation  for 
those  ratios  are  given  In  Table  9-3- 


TtMe  9-2 

Optratlont  /  ♦  K)^  for  Ordlnoiy  Explicit 


Problem 

ID 

2k  IE  lA 

5* 

10 

IF 

ZB 

IB  evg. 

c.v 

1 

1.5fl  1.32  .62 

•  59 

.AO 

.70 

.3A 

2,22  1.02 

.6 

T«bl«  9>3 

Op«mione  /  for  Othor  Runs 


Alforit^im 

Pomi 

Basis 

Run 

Average 

na 

standard 

S 

10 

1.36 

PNZ 

OrtAtest-ohenge 

1 

s 

s 

lA 

Ordinary 

product 

s 

56 

.73 

Synnetrlc 

standard 

F 

ko 

2.57 

CMlnary 

product 

F 

55 

.73 

<D  O' 
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Sinco  In  practice  K  Is  usually  1,  we  can  say  that 
around  operations  are  required  to  solve  a  linear 

procrammlng  problem.  A  rou"h  minimum  for  problems  of  no 
more  tlian  some  100  constraints  Is  0.3  ,  and  2  is 

a  rouch  maximum  for  smaller  problems .  A  count  of  more  than 
3  indicates  an  uncommonly  hard  problem  or  a  rather 
poor  algorithm. 


X.  CONCLUSION 


Three  kinds  of  data  have  been  used  above:  Iterations, 
operations,  and  passes.  We  have  come  to  the  view  that 
Iterations  alone  Is  the  least  Informative:  on  the  one 
hand,  the  operation  count  measures  the  total  work  of  a 
routine,  and  on  the  other  passes  measure  the  amount  of  data 
handled.  Of  course,  except  for  those  of  Sec.  VII,  cub- 
optimization  Is  not  used  In  any  of  the  routines  studied, 
so  that  In  general  the  number  of  passes  Is  equal  to  the 
number  of  Iterations,  which  Is  the  number  we  usually  cite. 

The  results  of  Sec.  IV  show  that  use  of  a  full  basis 
will  reduce  the  Iterations  taken  In  Phase  One.  (in  Sec. 

IX,  however,  we  found  that  It  Is  of  little  value  In  re¬ 
ducing  the  operation  count  for  the  most  efficient  procedure.) 
It  appears  that  there  Is  no  excuse  for  using  am  entirely 
artificial  basis. 

In  Sec.  V  we  failed  to  find  any  measure  of  Infeasl- 
blllty  with  which  to  conduct  fhacc  One  '.'hVh  wn-rks  better 
than  the  ordinary  measure— the  sum  of  all  the  Infeaslbllltles . 

The  results  of  Sec.  VI  show  the  positive-normalized 
procedures  beat  In  terms  of  Iteration  count,  and  that  the 
full  basis  Is  good  for  the  overall  problem.  The  first 
conclusion  Is  consistent  with  the  Interesting  results  of 
Kuhn  and  Quandt, who  have  experimented  with  several 
pivot-column  selection  procedures  on  a  large  number  of 
randomly-generated  linear  programming  problems  of  special 
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type  liavinc  up  to  23  constraints.  In  the  only  place  where 
their  results  can  be  matched  with  ours,  we  agree  in 
ordering  these  procedures  in  increasing  effectiveness  in 
iteration  count:  ordinary,  greatest-change,  and  positive- 
normalized.  Our  data  suggest  M  and  3M  as  bounds  for 
the  number  of  iterations  to  solve  a  problem  starting  from 
a  singleton  basis. 

The  extent  to  which  suboptimization  will  be  of  value 
In  a  routine  depends  considerably  on  how  Its  data-handllng 
is  organized.  Section  Vil  shows  that  It  can  be  used  with 
little  harm  and  under  some  circumstances  with  benefit  to 
the  total  computational  labor. 

The  comparison  of  operations  per  Iteration  In  Sec. 

VIII  shows  pretty  definitely  that  the  order  of  the  three 
main  forms  of  the  simplex  method  In  Increasing  efficiency 
is;  standard,  explicit,  product.  The  fact  that  those 
algorithms  which  are  better  than  the  ordinary  In  Iterations 
need  data  which  are  conveniently  obtained  only  In  the 
standard  form  makes  them  less  attractive  from  the  point  of 
view  of  operation  count;  Sec.  IX  .:hows  that  the  ordlneuy 
algorithm  In  product  form  leads  the  rest.  There  ar>e  other 
considerations,  however,  for  general  uses  of  a  linear  pro- 
greunmlng  routine,  which  are  hard  to  evaluate  properly  but 
which  argue  for  the  standard  form:  In  that  form  most  of 
the  data  needed  for  the  usual  postoptlmal  analyses — re¬ 
duced  coots,  etc. — are  Immediately  available  and  need  not 
be  especially  calculated. 
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An  important  fact  about  the  product  form,  whose 
detailed  study  is  beyond  the  scope  of  this  report,  is  that 
the  product-form  Inverse  is  extremely  compact  for  problems 
of  low  density.  This  fact  has  considerable  bearing  on  the 
choice  of  a  routine  for  larger  problems.  SHARE  problem 
4A,  having  245  constraints,  can  be  solved  with  an  all-in- 
core  routlne^^^  for  the  IBM  JCSO,  which  has  32,768  words 
of  core.  A  3imll2U’  routine  using  the  explicit  form  would 
require  75,000  words,  and  using  the  standard  form,  118,000 
words . 

At  this  time,  we  feel  that  a  product-form  routine 
employing  the  ordinary  or  the  greatest-change  algorithm 
with  sub  ptlmlzatlon,  with  option  for  using  a  full  basis, 
will  pull  together  the  best  features  of  the  procedures  we 
have  studied  so  far. 

It  may  seem  disappointing  that  our  results  have  not 
allowed  a  more  decisive  ordering  of  the  proposals  studied. 

In  part,  of  course,  this  is  due  to  our  having  selected  the 
more  promising  possibilities  from  a  larger  number  of  candi¬ 
dates;  but  it  may  also  be  the  case  that,  as  llneo.r  programming 
Is  presently  understood,  it  is  not  possible  to  do  a  great 
deal  better  than  some  of  these  procedures  do.  A  linear  pro¬ 
gramming  method  has  two  parts:  find  the  optimal  basis,  and 
calculate  the  optimal  solution.  If  the  optimal  basis  were 
known.  It  would  still  In  the  general  case  require  seme 

operations  to  solve  the  linear  equations  thus  identified 


(although  a  product-form  method  would  do  much  better  for 
problems,  like  ours,  having  a  low  density  of  data).  Since 
some  of  our  procedures  do  tiie  whole  Job  In  about 
operations,  there  does  not  seem  to  be  an  enormous  amount 
of  room  for  Improvement. 
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